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— The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 GFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )IEI Responsive to communication(s) filed on 27 August 2010 . 
2a)D This action is FINAL. 2b)K This action is non-final. 

3) D An election was made by the applicant in response to a restriction requirement set forth during the interview on 

; the restriction requirement and election have been incorporated into this action. 

4) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

5) ^ Claim(s) 1-33 is/are pending in the application. 

5a) Of the above claim(s) 6-11, 1 7-22 and 26-33 is/are withdrawn from consideration. 

6) D Claim(s) is/are allowed. 

7) D Claim(s) is/are rejected. 

8) D Claim(s) is/are objected to. 

9) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

10)D The specification is objected to by the Examiner. 

1 1 )□ The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

12) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. § 119 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)DAII b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 

PTOL-326 (Rev. 03-1 1 ) Office Action Summary Part of Paper No./Mail Date 201 1 1 1 16 
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DETAILED ACTION 

• Claims 1-5, 12-16 and 23-25 are pending; claims 1,12 and 16 are amended; 
claims 6-11, 17-22 and 26-30 are withdrawn ; claims 31-33 are canceled. 



Continued Examination Under 37 CFR 1. 1 14 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 27 
August 2010 has been entered. 



Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

3. Claims 1 2-16 are rejected under 35 U.S.C. 1 01 because the claimed invention is 
directed to an article, which is a non -statutory subject matter. 



Claim Rejections - 35 USC §112 



4. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 
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5. Claims 1 -5, 1 2-16 and 23-25 are rejected under 35 U.S.C. 1 1 2, first paragraph, 
as failing to comply with the written description requirement. The claim(s) contains 
subject matter which was not described in the specification in such a way as to 
reasonably convey to one skilled in the relevant art that the inventor(s), at the time the 
application was filed, had possession of the claimed invention. The specification does 
not support the new amended limitation-"checking recorded time conditions in the 
execution scenario script to handle variations in playback speed." 



Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1 -5, 1 2-16 and 23-25 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Redstone Software, Inc. ("Software Automation & Testing", 
hereinafter, "Redstone") in view of Vinson et al. (US 2003/0172374 A1). 

a. Regarding claim 1 , Redstone discloses a computer-implemented method 
of automatically controlling execution of an application program by a computer system, 
the application program having a graphical user interface for displaying information on a 
display coupled to the computer system (page 2, "Eggplant") comprising: 
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capturing user input data and images (page 7, "capture mode"; page 8, "capture 
images of the target controls and screen locations") displayed by the graphical user 
interface on the display (page 8, figure) during a recording phase of execution of the 
application program by the computer system ("The invention is a software, which makes 
it readily apparent and inherent that a computer having a display is required to run it."); 

analyzing the captured user input data and displayed images to generate an 
execution scenario script during the recording phase (pages 8 and 12, "generating a 
script"), wherein the captured user input data is to be stored in a memory ("The 
invention is a software, which makes it readily apparent and inherent that a computer 
having a memory is required to run it."); 

generating simulated user input data based on the execution scenario during a 
playback phase of execution of the application program by the computer system and 
inputting the simulated user input data to the application program (page 4, "Scripts 
interact with the system-under-test at the same level that a real user would, keying off 
the same visual cues and sending the same inputs"); 

performing image analysis on images displayed by the graphical user interface 
on the display as a result of processing the simulated user input data during the 
playback phase and captured displayed images from the recording phase (page 4, 
"Scripted actions use advanced image-matching algorithms to locate elements of the 
interface to target"); and 
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automatically controlling execution of the application program by the computer 
system based at least in part on the image analysis (page 4, "Scripted actions use 
advanced image-matching algorithms to locate elements of the interface to target"). 

However, Redstone does not disclose checking recorded time conditions in the 
execution scenario script to handle variations in playback speed. 

Instead of Redstone, Vinson discloses checking recorded time conditions in the 
execution scenario script to handle variations in playback speed ("[s]uch controls may 
include buttons for stopping playback, starting playback at normal speed, controlling 
playback speed, controlling content playback direction, skipping forward or backward in 
the content stream, and the like" at f0351). 

Redstone and Vinson are combinable because both are related to the field of 
content display and control. 

At the time of the invention, it would have been obvious to one of the ordinary 
skill in the art to utilize the controls including buttons for stopping playback, starting 
playback at normal speed, controlling playback speed, controlling content playback 
direction, skipping forward or backward in the content stream, and the like of Vinson to 
Redstone's invention. 

The suggestion/motivation would have been to collect more reliable data through 
its use of private, non-invasive data collection techniques ... [and,] provide data which 
lends itself to more advanced, thorough, and privacy-secure analysis techniques ... 
[and] analyze data more accurately (Vinson; f0046). 
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b. Regarding claim 2, Redstone further discloses wherein performing image 
analysis comprising comparing the displayed images captured during the recording 
phase with displayed images from the playback phase (page 4, "advanced image- 
matching algorithm"). 

c. Regarding claim 3, Redstone further discloses wherein the user input data 
comprising at least one of keyboard input data and mouse input data (page 16, "driven 
by the mouse or the keyboard"). 

d. Regarding claim 4, Redstone further discloses wherein analyzing the 
captured user input data and displayed images in the recording phase comprising 
identifying actions of the application program corresponding to the captured user input 
data (Page 7, "For the user to interact remotely with system using script, it is readily 
apparent and inherent that the analyzing process will identify actions of the application 
program corresponding to the captured user input data."). 

e. Regarding claim 5, the combination of Redstone and Vinson discloses 
using a script, but not any details which markup language is used. However, the 
applicant does not disclose any advantage or reason why XML is used as the markup 
language for the script. Especially, the applicant only discloses that XML can be used 
as one of the high level language for the script in the abstract. So, using a XML for the 
script will be a design choice. 

It would have obvious in one of the ordinary skill in the art to use XML as a 
markup language for the script of the combination of Redstone and Vinson because 
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XML is one of the widely used markup language and will provide simplicity, generality, 
and usability for applications. 

Regarding claims 1 2-16, claims 1 2-1 6 are analogous and correspond to claim 1 - 
5, respectively. See rejection of claims 1-5 for further explanation. 

Regarding claims 23-25, claims 23-25 are analogous and correspond to claim 1 - 
2 and 4, respectively. See rejection of claims 1-2 and 4 for further explanation. 

Conclusion 

8. No claims are allowed. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JOHN LEE whose telephone number is (571 )272-9554. 
The examiner can normally be reached on Monday - Friday (Alt.) 7:30 a.m. - 5:00 p.m.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on (571 ) 272-741 3. The fax phone number 
for the organization where this application or proceeding is assigned is 571 -273-8300. 
Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 
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Customer Service Representative or access to the automated information system, call 
800-786-91 99 (IN USA OR CANADA) or 571 -272-1 000. 



/JOHN W. LEE/ 

Primary Examiner, Art Unit 2624 



